Skip to content

[jsweep] Clean validate_lockdown_requirements.cjs#39498

Merged
pelikhan merged 7 commits into
mainfrom
signed/jsweep/validate-lockdown-requirements-628d174c9a4092dd
Jun 16, 2026
Merged

[jsweep] Clean validate_lockdown_requirements.cjs#39498
pelikhan merged 7 commits into
mainfrom
signed/jsweep/validate-lockdown-requirements-628d174c9a4092dd

Conversation

@github-actions

@github-actions github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

What

Centralise the three lockdown-validation error messages in validate_lockdown_requirements.cjs into a dedicated template module, and tighten the accompanying tests.

Why

The three error-message strings were constructed inline as raw string concatenations, making them hard to read, maintain, or test independently. Extracting them into named templates:

  • makes the messages discoverable as a unit,
  • ensures URLs and compile commands are defined once in a shared context object rather than duplicated,
  • keeps the validation logic focused on control-flow rather than string assembly, and
  • enables content-level test assertions without coupling tests to the exact call site.

Additionally, the repeated setOutputsetFailedthrow pattern appeared three times in the implementation; deduplicating it into a failWithError helper reduces the blast-radius of any future changes to that exit sequence.

How

1. New template module — validate_lockdown_requirements_templates.cjs (added)

Exports one render function per error scenario:

Export Scenario
renderLockdownTokenErrorMessage() Lockdown mode enabled, no custom token present
renderPublicStrictModeErrorMessage() Public repo compiled without --strict
renderPullRequestTargetErrorMessage() pull_request_target trigger on a public repo

Each function calls renderTemplate(TEMPLATE, TEMPLATE_CONTEXT) from messages_core.cjs. The shared TEMPLATE_CONTEXT holds auth_docs_url, security_docs_url, and strict_compile_command, so doc URLs and the compile command are defined in exactly one place.

2. Implementation cleanup — validate_lockdown_requirements.cjs (modified)

  • Removed unused ERR_VALIDATION import from error_codes.cjs.
  • Extracted inner failWithError(message) helper that performs setOutput("lockdown_check_failed", "true") + setFailed(message) + throw, eliminating three identical call-site copies.
  • Replaced three inline multi-line string constructions with calls to the corresponding template render functions from the new module.
  • Moved the template-module require to the top of the file (style).

3. Test improvements — validate_lockdown_requirements.test.cjs (modified)

  • Added assertions for setOutput side-effects on successful-validation paths (ensures lockdown_check_failed is not set on the happy path).
  • Added a test exercising simultaneous presence of multiple custom-token configurations.
  • Added content-level assertions on the rendered output of all three error message templates.
  • Removed a redundant pull_request_target assertion that duplicated coverage already provided by more focused cases.
  • Refocused remaining assertions on observable behaviour rather than exact string literals, making tests resilient to future message wording tweaks.

Files changed

File Change Impact
actions/setup/js/validate_lockdown_requirements_templates.cjs added Medium — new module; no runtime behaviour change
actions/setup/js/validate_lockdown_requirements.cjs modified Medium — implementation refactor; identical observable behaviour
actions/setup/js/validate_lockdown_requirements.test.cjs modified Low — test-only; broader coverage

Notes

  • Breaking change: No.
  • Runtime behaviour: Unchanged — error messages rendered by renderTemplate are semantically identical to the previously inlined strings; only the construction mechanism differs.
  • The base of this branch includes 52122d075 (narrow POC scope to PR outcomes and safe-output outcomes only), which is unrelated to the lockdown refactor and will be absorbed by the merge.

Generated by PR Description Updater for issue #39498 ·

github-actions Bot and others added 2 commits June 16, 2026 05:46
- Remove unused ERR_VALIDATION import
- Extract failWithError helper to eliminate repeated setOutput+setFailed+throw pattern (DRY)
- Use template literals with \n for multi-line error messages instead of string concatenation
- Add 6 new tests: setOutput side-effects, multiple tokens, error message newline formatting

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@pelikhan pelikhan marked this pull request as ready for review June 16, 2026 14:11
Copilot AI review requested due to automatic review settings June 16, 2026 14:11
@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Test Quality Sentinel completed test quality analysis.

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

PR Code Quality Reviewer completed the code quality review.

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #39498 does not have the 'implementation' label (has_implementation_label=false) and has 0 new lines of code in business logic directories (requires_adr_by_default_volume=false, default_business_additions=0, threshold=100).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR performs a small cleanup/refactor of the actions/setup/js/validate_lockdown_requirements.cjs runtime validator (used from github-script), and strengthens its Vitest coverage to ensure behavior and error formatting remain correct.

Changes:

  • Removed an unused ERR_VALIDATION import from validate_lockdown_requirements.cjs.
  • Extracted a failWithError(message) helper to de-duplicate the repeated failure pattern (setOutput + setFailed + throw).
  • Added additional tests to validate setOutput side effects, multi-token configurations, and newline formatting in error messages.
Show a summary per file
File Description
actions/setup/js/validate_lockdown_requirements.cjs Removes unused import, factors repeated failure logic into a helper, and switches error message construction to newline-containing strings.
actions/setup/js/validate_lockdown_requirements.test.cjs Expands coverage to verify no success-path setOutput, multi-token success behavior, and newline presence in error messages.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /improve-codebase-architecture and /tdd — approving with two non-blocking suggestions.

📋 Key Themes & Highlights

Key Themes

  • Template literal chaining: The \\n\n semantic fix is real and correct, but the strings are still built by concatenating individual template literals. A single template literal (or a true multiline one) would complete the job.
  • .toThrow() consistency: The three new formatting tests use bare .toThrow() while the existing suite already passes a message substring to .toThrow(...). Small gap worth closing.

Positive Highlights

  • @returns {never} JSDoc on failWithError is precise — TypeScript will use this for control-flow narrowing.
  • failWithError DRY extraction eliminates three identical three-liner blocks cleanly.
  • ✅ The \\n\n migration is a genuine semantic bug fix (old code emitted literal backslash-n text), and the new error message formatting describe block provides direct proof.
  • ✅ Success-path setOutput side-effect tests (expect(mockCore.setOutput).not.toHaveBeenCalled()) are excellent regression guards.
  • ✅ Dead ERR_VALIDATION import removed, eliminating an unnecessary coupling to error_codes.cjs.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 217.6 AIC · ⌖ 14 AIC · ⊞ 29.4K

Comment thread actions/setup/js/validate_lockdown_requirements.cjs Outdated

expect(() => {
validateLockdownRequirements(mockCore);
}).toThrow();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] .toThrow() without an argument does not verify which error is thrown — inconsistent with the existing suite (line 301 uses .toThrow("not compiled with strict mode")).

💡 Match the specificity of existing tests

The pre-existing test block uses:

}).toThrow("not compiled with strict mode");

Applying the same pattern here makes all three new formatting tests consistent and catches regressions in the thrown message, not just the fact that something was thrown:

// lockdown test
}).toThrow("no custom GitHub token is configured");

// strict mode test
}).toThrow("not compiled with strict mode");

// pull_request_target test
}).toThrow("pwn request");

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Clean, correct refactor. No blocking issues.

What was examined

failWithError extraction — Correct DRY helper. core.setFailed does not throw in the Actions runtime, so the subsequent throw new Error(message) always executes. The @returns {never} JSDoc annotation is valid and aids TypeScript control-flow analysis under // @ts-check``.

\\n\n behavioral fix — Old code used "text\\\\n" in regular string literals, producing literal backslash-n in the string value. New code uses `text\n` in template literals, producing real newline characters. @actions/core properly URL-encodes \n%0A in ::error:: annotations. This is a genuine bug fix that improves log readability.

New tests — The three added describe blocks (setOutput side effects, multiple tokens, error message formatting) validate concrete behaviors: no false-positive output on success paths, correct || token logic, and the \n fix round-trip. Scope is appropriate for unit tests.

🔎 Code quality review by PR Code Quality Reviewer · 340.5 AIC · ⌖ 13.4 AIC · ⊞ 17.3K

core.setFailed(errorMessage);
throw new Error(errorMessage);
failWithError(
`Lockdown mode is enabled (lockdown: true) but no custom GitHub token is configured.\n` +

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot refactor to template file and renderTemplate

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in commit 6c9ce2b. I moved the lockdown/strict/pull_request_target error text into a dedicated template module and render it via renderTemplate.

core.setFailed(errorMessage);
throw new Error(errorMessage);
failWithError(
`This workflow is running on a public repository but was not compiled with strict mode.\n` +

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot refactor to template file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in commit 6c9ce2b. The strict-mode public-repo failure message now comes from the new template module and is rendered through renderTemplate.

core.setFailed(errorMessage);
throw new Error(errorMessage);
failWithError(
`This workflow is triggered by the pull_request_target event on a public repository.\n` +

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot refactor to template file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in commit 6c9ce2b. The pull_request_target public-repo failure message was moved to the template module and rendered through renderTemplate.

@github-actions

Copy link
Copy Markdown
Contributor Author

🧪 Test Quality Sentinel Report

⚠️ Test Quality Score: 60/100 — Acceptable

Analyzed 6 test(s): 3 design, 3 implementation, 0 guideline violations.

📊 Metrics & Test Classification (6 tests analyzed)
Metric Value
New/modified tests analyzed 6
✅ Design tests (behavioral contracts) 3 (50%)
⚠️ Implementation tests (low value) 3 (50%)
Tests with error/edge cases 3 (50%)
Duplicate test clusters 1
Test inflation detected No (1.82:1 ratio)
🚨 Coding-guideline violations 0
Test File Classification Issues Detected
should not set lockdown_check_failed output on a fully successful run validate_lockdown_requirements.test.cjs ✅ Design
should not set lockdown_check_failed when lockdown is enabled with token and repo is private validate_lockdown_requirements.test.cjs ✅ Design
should pass when all three tokens are configured simultaneously validate_lockdown_requirements.test.cjs ✅ Design ⚠️ info-log assertions are implementation-level
should include newlines in lockdown error message for readability validate_lockdown_requirements.test.cjs ⚠️ Implementation Asserts exact message wording and \n formatting
should include newlines in strict mode error message for readability validate_lockdown_requirements.test.cjs ⚠️ Implementation Asserts exact message wording and \n formatting
should include newlines in pull_request_target error message for readability validate_lockdown_requirements.test.cjs ⚠️ Implementation Asserts exact message wording and \n formatting

Go: 0; JavaScript: 6 (*.test.cjs). No other languages detected.

⚠️ Flagged Tests — Requires Review (3 issues)

should include newlines in lockdown error message for readability (actions/setup/js/validate_lockdown_requirements.test.cjs) — ⚠️ Implementation: asserts toContain("\n") plus exact option strings. The newline check tests internal message formatting, not a user-facing behavioral contract — it breaks on any whitespace refactoring even when validation behavior is unchanged. Suggested fix: remove the toContain("\n") assertion; retain toContain("GH_AW_GITHUB_TOKEN (recommended)") and toContain("GH_AW_GITHUB_MCP_SERVER_TOKEN (alternative)") as actionable-guidance contracts.

should include newlines in strict mode error message for readability (actions/setup/js/validate_lockdown_requirements.test.cjs) — ⚠️ Implementation: same pattern. toContain("\n") is a formatting assertion with no behavioral value. Suggested fix: drop the newline assertion, keep toContain("gh aw compile --strict") which genuinely tests that the actionable command appears in the error.

should include newlines in pull_request_target error message for readability (actions/setup/js/validate_lockdown_requirements.test.cjs) — ⚠️ Implementation + Duplication: same setFailed.mock.calls[0][0] + toContain("\n") pattern as the two tests above, forming a 3-test duplicate cluster. Suggested fix: remove the \n assertions across all three; if verifying multi-error-path message content is the goal, a parameterised helper (e.g. test.each) would eliminate the duplication and make behavioral intent clearer.

Verdict

Check failed. 50% implementation tests (threshold: 30%). The three error message formatting tests verify internal \n presence and exact message wording rather than behavioral contracts. The content assertions (toContain("gh aw compile --strict"), etc.) do carry real behavioral value — removing only the newline assertions and de-duplicating the pattern would be sufficient to bring this PR above the threshold.

References: §27623742541

🧪 Test quality analysis by Test Quality Sentinel · 414.3 AIC · ⌖ 19.1 AIC · ⊞ 26.5K ·

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Test Quality Sentinel: 60/100. 50% of new tests are classified as low-value implementation tests, exceeding the 30% threshold. The three error message formatting tests assert on \n presence and exact message wording (internal formatting) rather than behavioral contracts. Please review the flagged tests in the comment above and remove the newline assertions — the content assertions are valuable and should be kept.

Copilot AI and others added 2 commits June 16, 2026 14:45
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI requested a review from pelikhan June 16, 2026 14:49
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot run pr-finisher skill

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

@copilot run pr-finisher skill

Ran the pr-finisher flow and applied the remaining in-scope test feedback in 625bc90 (behavior-focused error assertions with specific .toThrow(...) checks). Local validation is green (make fmt, make lint, make test-unit, make test, make recompile).

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan pelikhan merged commit 54d56aa into main Jun 16, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants